Live Trace Visualization for System and Program Comprehension in Large Software Landscapes

The number of software systems in modern enterprise architectures is constantly increasing and thus also the complexity of such software landscapes. In addition, the knowledge of the internal behavior and utilization often gets lost. Software visualization can provide a solution to these challenges. For instance, UML or the city metaphor are established concepts. We utilize these concepts to visualize the communication and entities in a software landscape to ease system comprehension. Our ExplorViz approach visualizes the communication taking place on both the landscape level and the system level. In this paper, we present our PhD project: live trace visualization for system and program comprehension in large software landscapes. For this purpose, our research questions and a sketch of our approach, named ExplorViz, are described. Furthermore, this paper illustrates ideas for the planned evaluation of our approach

Live Trace Visualisierung für System- und Programmverständnis in großen Softwarelandschaften

In many enterprises, the number of deployed applications is constantly increasing. Those applications - often several hundreds - form large software landscapes. The comprehension of such landscapes is frequently impeded due to, for instance, architectural erosion, personnel turnover, or changing requirements. Furthermore, events such as performance anomalies can often only be understood in correlation with the states of the applications. Therefore, an efficient and effective way to comprehend such software landscapes in combination with the details of each application is required. In this thesis, we introduce a live trace visualization approach to support system and program comprehension in large software landscapes. It features two perspectives: a landscape-level perspective using UML elements and an application-level perspective following the 3D software city metaphor. Our main contributions are 1) an approach named ExplorViz for enabling live trace visualization of large software landscapes, 2) a monitoring and analysis approach capable of logging and processing the huge amount of conducted method calls in large software landscapes, and 3) display and interaction concepts for the software city metaphor beyond classical 2D displays and 2D pointing devices. Extensive lab experiments show that our monitoring and analysis approach elastically scales to large software landscapes while imposing only a low overhead on the productive systems. Furthermore, several controlled experiments demonstrate an increased efficiency and effectiveness for solving comprehension tasks when using our visualization. ExplorViz is available as open-source software on www.explorviz.net. Additionally, we provide extensive experimental packages of our evaluations to facilitate the verifiability and reproducibility of our results.In vielen Unternehmen nimmt die Anzahl der eingesetzten Anwendungen stetig zu. Diese Anwendungen - meist mehrere hunderte - bilden große Softwarelandschaften. Das Verständnis dieser Softwarelandschaften wird häufig erschwert durch, beispielsweise, Erosion der Architektur, personelle Wechsel oder sich ändernde Anforderungen. Des Weiteren können Ereignisse wie Performance-Anomalien häufig nur in Verbindung mit den Anwendungszuständen verstanden werden. Deshalb wird ein möglichst effizienter und effektiver Weg zum Verständnis solcher Softwarelandschaften in Verbindung mit den Details jeder einzelnen Anwendung benötigt. In dieser Arbeit führen wir einen Ansatz zur live Trace Visualisierung zur Unterstützung des System- und Programmverständnisses von großen Softwarelandschaften ein. Dieser verwendet zwei Perspektiven: eine Landschaftsperspektive mit UML Elementen und eine Applikationsperspektive, welche der 3D Softwarestadtmetapher folgt. Unsere Hauptbeiträge sind 1) ein Ansatz, genannt ExplorViz, um live Trace Visualisierung von großen Softwarelandschaften zu ermöglichen, 2) ein Überwachungs- und Analyseansatz, welcher in der Lage ist die große Anzahl an Methodenaufrufen in einer großen Softwarelandschaft aufzuzeichnen und zu verarbeiten und 3) Anzeige- und Interaktionskonzepte für die Softwarestadtmetapher, welche über klassische 2D Anzeige und 2D Eingabegeräten hinausgehen. Umfassende Laborexperimente zeigen, dass unser Überwachungs- und Analyseansatz für große Softwarelandschaften elastisch skaliert und dabei nur einen geringen Overhead auf den Produktivsystemen erzeugt. Des Weiteren demonstrieren mehrere kontrollierte Experimente eine gesteigerte Effizienz und Effektivität beim Lösen von Verständnisaufgaben unter Verwendung unserer Visualisierung. ExplorViz ist als Open Source Anwendung verfügbar unter www.explorviz.net. Zusätzlich stellen wir umfangreiche Pakete für unsere Evaluierungen zur Verfügung um die Nachvollziehbarkeit und Wiederholbarkeit unserer Ergebnisse zu ermöglichen

Research Perspective on Supporting Software Engineering via Physical 3D Models

Building architects, but also civil or mechanical engineers often build from their designs physical3D models for a better presentation, comprehension, and communication among stakeholders. Software engineers usually create visualizations of their software designs as digital objects to be presented on a screen only. 3D software visualization metaphors, such as the software city metaphor, provide a basis for exporting those on-screen software visualizations into physical models. This can be achieved by 3D-printers to transfer the advantages of real, physical, tangible architecture models from traditional engineering disciplines to software engineering. We present a new research perspective of using physical models to support software engineering. Furthermore, we describe four envisioned usage scenarios for physical models which provide a plethora of new research topics. As proof of concept, we investigate the first usage scenario by evaluating the impact of using physical models on program comprehension in teams through a first controlled experiment. Our experiment reveals that the usage of physical models has a diverging influence. However, they improve the team-based program comprehension process for specific tasks by initiating gesture-based interaction. Therefore, the experiment shows that physical models can provide a promising future research direction

CDOXplorer: Simulation-based genetic optimization of software deployment and reconfiguration in the cloud

Migrating existing enterprise software to cloud platforms involves the comparison of various cloud deployment options (CDOs). A CDO comprises a combination of a specific cloud environment, deployment architecture, and runtime reconfiguration rules for dynamic resource scaling. Our simulator CDOSim can evaluate CDOs, e.g., regarding response times and costs. However, the design space to be searched for well-suited solutions is very large. In this paper, we approach this optimization problem with the novel genetic algorithm CDOXplorer. It uses techniques of the search-based software engineering field and simulations with CDOSim to assess the fitness of CDOs. An experimental evaluation that employs, among others, the cloud environments Amazon EC2 and Microsoft Windows Azure, shows that CDOXplorer can find solutions that surpass those of other state-of-the-art techniques by up to 60\%. Our experiment code and data and an implementation of CDOXplorer are available as open source software

Performance Analysis of Legacy Perl Software via Batch and Interactive Trace Visualization

Performing an analysis of established software usually is challenging. Based on reverse engineering through dynamic analysis, it is possible to perform a software performance analysis, in order to detect performance bottlenecks or issues. This process is often divided into two consecutive tasks. The first task concerns the monitoring of the legacy software, and the second task covers analysing and visualizing the results. Dynamic analysis is usually addressed via trace visualization, but finding an appropriate representation for a specific issue still remains a great challenge. In this paper we report on our performance analysis of the Perl-based open repository software EPrints, which has now been continuously developed for more than fifteen years. We analyse and evaluate the software using the Kieker monitoring framework, and apply and combine two types of visualization tools, namely Graphviz and Gephi. More precisely, we employ Kieker to reconstruct architectural models from recorded monitoring data, based on dynamic analysis, and Graphviz respectively Gephi for further analysis and visualization of our monitoring results. We acquired knowledge of the software through our instrumentation and analysis via Kieker and the combined visualization of the two aforementioned tools. This allowed us, in collaboration with the EPrints development team, to reverse engineer their software EPrints, to give new and unexpected insights, and to detect potential bottlenecks

Cloud user-centric enhancements of the simulator cloudsim to improve cloud deployment option analysis

Abstract. Cloud environments can be simulated using the toolkit CloudSim. By employing concepts such as physical servers in datacenters, virtual machine allocation policies, or coarse-grained models of deployed software, it focuses on a cloud provider perspective. In contrast, a cloud user who wants to migrate complex systems to the cloud typically strives to find a cloud deployment option that is best suited for its sophisticated system architecture, is interested in determining the best trade-off between costs and performance, or wants to compare runtime reconfiguration plans, for instance. We present significant enhancements of CloudSim that allow to follow this cloud user perspective and enable the frictionless integration of fine-grained application models that, to a great extent, can be derived automatically from software systems. Our quantitative evaluation demonstrates the applicability and accuracy of our approach by comparing its simulation results with actual deployments that utilize the cloud environment Amazon EC2

Elastic Application-Level Monitoring for Large Software Landscapes in the Cloud

Part 1: Research TrackInternational audienceApplication-level monitoring provides valuable, detailed insights into running applications. However, many approaches often only employ a single analysis application. This analysis application may become a performance bottleneck when monitoring several programs resulting in reduced monitoring quality or violated service level agreements of the monitored applications.We present an approach for elastic, distributed application-level monitoring for large software landscapes consisting of several hundreds of applications by utilizing cloud computing. Our approach dynamically inserts and removes worker levels to circumvent overloading the analysis master application without interrupting or pausing the actual live analysis of the monitored data. To evaluate our approach, we conduct an experiment in which we generate load – following a real workload pattern – on web applications in a 24 hour experiment.In our experiment, 160 monitored JPetStore instances generate roughly 20 million analyzed method calls per second in the peak. Furthermore, two worker levels are dynamically started and removed in line with the imposed workload on the monitored applications. The experiment shows that our monitoring approach is capable of live analyzing several millions of monitored method calls per second without overloading the analysis master application

ExplorViz: Visual Runtime Behavior Analysis of Enterprise Application Landscapes

Enterprise application landscapes are complex and hard to manage systems. Enterprise models abstract from this complexity and seek to capture relevant information to cope with business requirements, i.e., increase flexibility while reducing costs of IT. Recent automated approaches focus on utilizing existing information sources. We identified two issues in current approaches. First, in line with these approaches we observe that enterprise models often get outdated and, thus, are an unreliable basis for decision making. Second, these approaches lack detail and after larger transformations commonly the entire application landscape exhibits an altered load profile. \ \ In this paper, we present how ExplorViz can be utilized to ensure consistency between an enterprise model and the actual information systems. We exemplify our approach via modeling the application landscape of the Kiel Data Management Infrastructure at the Helmholtz Centre for Ocean Research Kiel (GEOMAR). In our scenario, we depict an application landscape and exemplary drill-down to our EPrints productive installation to the source-code level. We explain the importance of underlying concepts and features, which help to optimize the responsiveness of an entire application landscape based on runtime information

Experimental Data for: Hierarchical Software Landscape Visualization for System Comprehension: A Controlled Experiment

<p>In many enterprises the number of deployed applications is constantly increasing. Those applications - often several hundreds - form large software landscapes. The comprehension of such landscapes is frequently impeded due to, for instance, architectural erosion, personnel turnover, or changing requirements. Therefore, an efficient and effective way to comprehend such software landscapes is required. The current state of the art often visualizes software landscapes via flat graph-based representations of nodes, applications, and their communication.</p> <p>In our ExplorViz visualization, we introduce hierarchical abstractions aiming at solving typical system comprehension tasks fast and accurately for large software landscapes. To evaluate our hierarchical approach, we conduct a controlled experiment comparing our hierarchical landscape visualization to a flat, state-of-the-art visualization. In addition, we thoroughly analyze the strategies employed by the participants and provide a package containing all our experimental data to facilitate the verifiability, reproducibility, and further extensibility of our results.</p> <p>We observed a statistically significant increase of 14 % in task correctness of the hierarchical visualization group compared to the flat visualization group in our experiment. The time spent on the system comprehension tasks did not show any significant differences. The results backup our claim that our hierarchical concept enhances the current state of the art in landscape visualization.</p> <p>This package contains our experimental data.</p